Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(schema-compiler): Move transpiling to worker threads (under the flag) #9188

Merged
merged 9 commits into from
Feb 6, 2025

Conversation

KSDaemon
Copy link
Member

@KSDaemon KSDaemon commented Feb 5, 2025

Parsing and transpiling huge data modes takes a significant amount of time and blocks the node event loop.
This PR tries to improve this.

Some charts are below. It has ~300 model files with ~400 members each.

Before:
Screenshot 2025-02-06 в 00 43 58

After: Non-blocking sequential processing using 1 thread
Screenshot 2025-02-06 в 01 01 22

After: Non-blocking parallel processing using N threads (default to the number of CPU's minus one, in my case 5)
Screenshot 2025-02-06 в 01 01 29

Check List

  • Tests have been run in packages where changes made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet
  • Docs have been added / updated if required

@KSDaemon KSDaemon marked this pull request as ready for review February 5, 2025 23:15
@KSDaemon KSDaemon requested review from a team as code owners February 5, 2025 23:15
@paveltiunov paveltiunov merged commit 7451452 into master Feb 6, 2025
54 checks passed
@paveltiunov paveltiunov deleted the feat/compilation-worker-threads branch February 6, 2025 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants